feat(ci): automated performance regression detection in CI pipeline - #179
Merged
elizabetheonoja-art merged 2 commits intoAug 20, 2026
Conversation
Adds a system-wide performance regression gate to CI: - Core detection logic in src/utils/performanceRegression.ts with 24 unit tests: nearest-rank percentiles, budget evaluation, and baseline-relative regression detection with tolerance. - Playwright benchmark (tests/performance/performance.spec.ts) that measures API critical paths (P99 < 100ms budget) and page TTFB/LCP, writing raw samples for analysis. - scripts/analyze-performance.ts gates CI on budget breaches/regressions and emits an HTML dashboard, machine-readable report, and PR comment; scripts/update-performance-baseline.ts refreshes the committed baseline. - New performance-regression.yml workflow: runs benchmarks on PRs, posts a findings comment, uploads report artifacts, and refreshes the baseline on main (supports blue-green/canary analysis via the same baseline mechanism). - Docs: architecture (docs/performance-regression-detection.md) and runbook (docs/runbooks/performance-regression.md). Closes Utility-Protocol#171
Two CI failures on the PR: 1. The performance benchmark asserted response.ok() on /api/runtime-config/ audit, but that endpoint deliberately returns 503 when critical runtime config drifts (env vars unset in CI). Measure round-trip latency regardless of HTTP status so the sample stays valid; network-level failures still throw and fail the test. 2. npm audit and OSV scans were failing on pre-existing high-severity vulnerabilities (next, @stellar/stellar-sdk's axios, sharp, postcss, undici, etc.). Upgraded next to ^16.3.1 and @stellar/stellar-sdk to ^16.2.0 and ran npm audit fix, clearing all 10 reported vulnerabilities. Verified with tsc, eslint, the full 678-test unit suite, a production build, and a local OSV scan (no issues found).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated Performance Regression Detection in CI Pipeline
Closes #171
Problem
Performance regressions can silently slip into the frontend — a heavier bundle, a slower API handler, or an expensive render path degrades P99 latency of critical paths without breaking any functional test. This PR adds an automated performance regression detection pipeline that runs in CI and blocks merges when critical paths regress.
What's included
1. Solution architecture & design
docs/performance-regression-detection.md— architecture doc covering the measurement → analysis → gate flow, budgets, baseline lifecycle, monitoring/alerting/dashboards, and blue-green/canary analysis support.2. Core logic with comprehensive tests
src/utils/performanceRegression.ts— pure, deterministic detection logic: nearest-rank percentiles, budget evaluation, and baseline-relative regression detection with configurable tolerance.tests/unit/performanceRegression.test.ts— 24 unit tests (100% statement/function/line coverage, 92% branch coverage).3. Monitoring, alerting, and dashboards
tests/performance/performance.spec.ts— Playwright benchmark measuring API critical paths (/api/runtime-config/audit,/api/rate-limit) with a P99 < 100ms budget plus page TTFB/LCP for/and/export.scripts/analyze-performance.ts— summarizes samples, evaluates budgets + baseline, emits an HTML dashboard (report.html), a machine-readable report (report.json), and a PR comment; exits non-zero to block the CI gate..github/workflows/performance-regression.yml— runs on PRs (posts findings comment + uploads artifacts) and refreshes the committed baseline onmain.4. Blue-green strategy and canary analysis
docs/TRACING_DEPLOYMENT.md.5. Runbooks and documentation
docs/runbooks/performance-regression.md— triage, remediation, budget tuning, and escalation.Gate behavior
passbudget-breachregressioninsufficient-dataVerification
npx tsc --noEmit✅npm run lint✅npx vitest --run— 678 tests passing (84 files) ✅